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Abstract 

We consider a sequence of transferable utility (TU) games where, at each time, 
the characteristic function is a random vector with realizations restricted to some set 
of values. The game differs from other ones in the literature on dynamic, stochastic 
or interval valued TU games as it combines dynamics of the game with an allocation 
protocol for the players that dynamically interact with each other. The protocol is 
an iterative and decentralized algorithm that offers a paradigmatic mathematical 
description of negotiation and bargaining processes. The first part of the paper 
contributes to the definition of a robust (coalitional) TU game and the development 
of a distributed bargaining protocol. We prove the convergence with probability 
1 of the bargaining process to a random allocation that lies in the core of the ro- 
bust game under some mild conditions on the underlying communication graphs. 
The second part of the paper addresses the more general case where the robust 
game may have empty core. In this case, with the dynamic game we associate a 
dynamic average game by averaging over time the sequence of characteristic func- 
tions. Then, we consider an accordingly modified bargaining protocol. Assuming 
that the sequence of characteristic functions is ergodic and the core of the average 
game has a nonempty relative interior, we show that the modified bargaining pro- 
tocol converges with probability 1 to a random allocation that lies in the core of the 
average game. 



1 Introduction 

Coalitional games with transferable utilities (TU) have been introduced by von Neumann 
and Morgenstern [27j. A coalitional TU game constitutes of a set of players, who can 
form coalitions, and a characteristic function that provides a value for each coalition. The 
value of a coalition can be thought of as a monetary value that can be divided among the 
members of the coalition according to some appropriate fairness allocation rule. 
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Coalitional TU games have been used to model cooperation in supply chain or inven- 
tory management applications [6l [10] , as well as in network flow applications to coordinate 
flows of goods, materials, or resources between different production/distribution sites [2]. 
Recently, coalitional games have also sparked much interest in communication networks. 
We refer an interested reader to tutorial [23] for an in-depth discussion on potential ap- 
plications in this fleld. 

In this paper, we consider a sequence of coalitional TU games for a flnite set of players. 
We assume that the game is played repeatedly over time thus generating a sequence of 
time varying characteristic functions. We refer to such a repeated game as dynamic 
coalitional TU game. In addition, we assume that a player need not necessarily observe 
the allocations of all the other players, but he can only observe the allocations of his 
neighbors, where the neighbors may change in time. 

In this setting, we consider robust and average TU games, and for each of these games 
we propose an allocation process to reach a solution of the dynamic game. More specifi- 
cally, we formulate a dynamic bargaining process as a dynamically changing constrained 
problem. Iterations are simply bargaining steps free of any concrete reward allocation. 
We assume that the reward allocation occurs only once at the end of the bargaining pro- 
cess, when the players have agreed on how to allocate rewards among themselves. Our 
main objective is to explore distributed agreement on solutions in the core of the game, 
where the players interact only with their neighbors. 

In particular, we consider bargaining protocols assuming that each player i obeys 
rationality and efficiency by deciding on an allocation vector which satisfies the value 
constraints of all the coalitions that include player i. This set is termed bounding set of 
player i. At every iteration, a player i observes the allocations of some of his neighbors. 
This is modeled using a directed graph with the set of players as the vertex set and a 
time- varying edge set composed of directed links {i,j) whenever player i observes the al- 
location vector proposed by player j at time t. We refer to this directed graph as players' 
neighbor- graph. Given a player's neighbor-graph, each player i negotiates allocations by 
adjusting the allocations he received from his neighbors through weight assignments. As 
the balanced allocation may violate his rationality constraints (it lies outside player z's 
bounding set), the player selects a new allocation by projecting the balanced allocation 
on his bounding set. We propose such bargaining protocols for solving both the robust 
and the average TU game. For each of these games, we use some mild assumptions on 
the connectivity of the players' neighbor-graph and the weights that the players use when 
balancing their own allocations with the neighbors' allocations. Assuming that the core 
of the robust game is nonempty, we show that our bargaining protocol converges with 
probability 1 to a common (random) allocation in the core. In the case when the core of 
the robust game is empty, we consider an average game that can provide a meaningful so- 
lution under some conditions on the sequence of the characteristic functions. Specifically, 
in this case, we consider a dynamic average game by averaging over time the sequence 
of characteristic functions. We then modify accordingly the bounding set and the associ- 
ated bargaining process by using the dynamic average game. This means that the value 
constraints are defined by using the time-averaged sequence rather than the sequence of 
random characteristic functions. Under the assumptions that the time-averaged sequence 
is ergodic and that the core of the average game has a nonempty relative interior, we 
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show that the players' allocations generated by our bargaining protocol converge with 
probability 1 to a common (random) allocation in the core of the average game. 

The main contributions of this paper are in the two-fold dynamic aspect of the games, 
and in the development and the analysis of distributed bargaining process among the 
players. The novelty in the dynamic aspect is in the consideration of games with time- 
varying characteristic functions and the use of time-varying local players interactions. 

The work in this paper is related to stochastic cooperative games [9l [2ll |25] . How- 
ever, we deviate from this stochastic framework in a number of aspects among which are 
the existence of a local players interactions captured by players' neighbor-graph and the 
presence of multiple iterations in the bargaining process, and the consideration of robust 
game. Bringing dynamical aspects into the framework of coalitional TU games is an ele- 
ment in common with papers [H [11]. However, unlike [H [11], the values of the coalitions 
in this paper are realized exogenously and no relation is assumed between consecutive 
samples. Dynamic robust TU games have been considered in [3] and [1] but for a con- 
tinuous time setting in the former work and for a centralized allocation process in the 
latter one. Convergence of allocation processes is a main topic in O [13]. There, rewards 
are allocated by a game designer repeatedly in a centralized manner and based on the 
current excess rewards of the coalitions (accumulated reward up to current time minus 
the value of the coalition). Our approach, however, differs from that of [13] as we resort 
to a decentralized scheme where the allocation process is the result of a bargaining pro- 
cess among the players with local interactions. Convergence of bargaining processes has 
also been explored under dynamic coalition formation [1] for a different dynamic model, 
where players decide both on which coalition to address and what payoff to announce. 
Coalitions form when announced payoffs are all met for all players in the coalition. Our 
bargaining procedure is different as it produces an agreement resulting in the formation 
of grand coalition, which is stable with respect to any sub-coalitions. 

The work in this paper is also related to the literature on agreement among multiple 
agents, where an underlying communication graph for the agents and balancing weights 
have been used with some variations flUi [T3] to reach an agreement on common decision 
variable, as well as in [Ml [13 [201 US] for distributed multi-agent optimization. 

This paper is organized as follows. In Section |2| we introduce the dynamic TU game, 
the robust game and the bargaining protocol for this game. We then motivate the game 
and give some preliminary results. In Section |3| we prove the convergence of the bargaining 
protocol to a point in the core of the robust game with probability 1. In Section |4| we 
introduce the dynamic average game and the modified bargaining protocol. We also 
prove the convergence of this protocol to an allocation in the core of the average game 
with probability 1. In Section |5j we report some numerical simulations to illustrate our 
theoretical study, and we conclude in Section |6| 

Notation. We view vectors as columns. For a vector x, we use Xi or [x]i to denote its 
ith coordinate component. For two vectors x and y, we use x < y {x < y) to denote 
Xi < yi {xi < yi) for all coordinate indices i. We let x' denote the transpose of a vector 
X, and denote its Euclidean norm. An n x n matrix A is row-stochastic if the matrix 
has nonnegative entries Ojj and o-ij = 1 for alH = 1, . . . , n. For a matrix A, we use 
aij or [A]ij to denote its ijth entry. A matrix A is doubly stochastic if both A and its 
transpose A' are row-stochastic. Given two sets U and S, we write U C S to denote that 
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f/ is a proper subset of 5". We use \S\ for the cardinality of a given finite set 5". 

We write Px[x] to denote the projection of a vector x on a set X, and we write 
dist(a:,X) for the distance from x to X, i.e., Px[x] = argmiuj^gx ll^; — ?/|| and dist(x,X) = 
\\x — Px[x]\\, respectively. Given a set X and a scalar A G M, the set XX is defined by 
AX = {\x \ X E X}. Given two sets X,Y C M", the set sum X + Y is defined by 
X + Y = {x + y\ xEX, y E Y}. Given a set N of players and a function rj : S ^ M. 
defined for each nonempty coalition S ^ N, we write < N,ri > to denote the transferable 
utility (TU) game with the players' set N and the characteristic function rj. We let rjs 
be the value ri{S) of the characteristic function rj associated with a nonempty coalition 
S" C X. Given a TU game < N,ri >, we use C{r]) to denote the core of the game, 

C{7]) = |x G MI^I Y^ieN^i = Vn, Y^ies^i ^ Vs for all nonempty S" C ivj . 



2 Dynamic TU Game and Robust Game 

In this section, we discuss the main components of a dynamic TU game and a robust 
game. We state some basic assumptions on these games and introduce a bargaining 
process conducted by the players to reach an agreement on their allocations. We also 
provide some examples of the TU games motivating our development and establish some 
basic results that we use later on in the convergence analysis of the bargaining process. 



2.1 Problem Formulation and Bargaining Process 

Consider a set of players N = {1, . . . ,n} and the set of all (nonempty) coalitions S ^ N 
arising among these players. Let m = 2" — 1 be the number of possible coalitions. We 
assume that the time is discrete and use t = 0, 1, 2, . . . to index the time slots. 

We consider a dynamic TU game, denoted < N, {v(t)} >, where {v(t)} is a sequence 
of characteristic functions. Thus, in the dynamic TU game < N, {v{t)} >, the players are 
involved in a sequence of instantaneous TU games whereby, at each time t, the instanta- 
neous TU game is < N,v{t) > with v{t) G M*" for all t > 0. Further, we let vs{t) denote 
the value assigned to a nonempty coalition S N in the instantaneous game < N,v{t) >. 
In what follows, we deal with dynamic TU games where each characteristic function v{t) 
is a random vector with realizations restricted to some set of values. We will consider the 
robust TU game in this section and Section |3| while in Section |4] we deal with the average 
TU game. 

In our development of robust TU game, we impose restrictions on each realization 
of v{t) for every t > 0. Specifically, we assume that the grand coalition value V]y{t) is 
deterministic for every t > 0, while the values vs{t) of the other coalitions S <Z N have a 
common upper bound. These conditions are formally stated in the following assumption. 

Assumption 1 There exists v^^"^ G such that for all t > 0, 

VN{t) = 

Vs{t) < f™^^ for all nonempty coalitions S G N. 
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We refer to the game < A^, y™*^ > as robust game. In the first part of this paper, we 
rely on the assumption that the robust game has a nonempty core. 

Assumption 2 The core of the robust game is not empty. 

An immediate consequence of Assumptions [T] and [2] is that the core C{v{t)) of the 
instantaneous game is always not empty. This follows from the fact that C (y™^^) C C {rj) 
for any rj satisfying rj^ = vf^^^ and 775 < v'^^^ for S G N, and the assumption that the 
core C(f™^^) is not empty. 

Throughout the paper, we assume that each player i is rational and efficient. This 
translates to each player i E N choosing his allocation vector within the set of allocations 
satisfying value constraints of all the coalitions that include player i. This set is referred 
to as the bounding set of player i and, for a generic game < N,ri >, it is given by 




Xi{7]) = {xeW'\yxj = r]N 



^ Xj > r]s for all S* C iV s.t. ie S> 
j&s ) 



Note that each Xi{ri) is polyhedral. 

In what follows, we find it suitable to represent the bounding sets and the core in 
alternative equivalent forms. For each nonempty coalition S" C A^, let eg G M" be the 
incidence vector for the coalition S, i.e., the vector with the coordinates given by 



1 if ieS, 
else. 



Then, the bounding sets and the core can be represented as follows: 

Xi{r]) = {xeW \ e'j^x = r/^v, e'gX > 7]s for all 5 C A with i e S}, (1) 



C{ri) = {x G M" I e'j^x = rj^, e'gX > rjs for all nonempty 5* C A}. (2) 

Furthermore, observe that the core C{ri) of the game coincides with the intersection of 
the bounding sets Xi{ri) of all players z G A = {1, . . . ,n}, i.e., 

C(r/) = nr=iA,(r/). (3) 

We now discuss the bargaining protocol where repeatedly over time each player i G 
A submits an allocation vector that the player would agree on. The allocation vector 
proposed by player i at time t is denoted by G M", where the jth component x*(t) 
represents the amount that player i would assign to player j. To simplify the notation in 
the dynamic game < A, {v(t)} >, we let Xi(t) denote the bounding set of player i for the 
instantaneous game < N,v(t) >, i.e., for alH G A and t > 0, 

Xi{t) = Ix \ ^Xj = VN{t), ^ ^s{t) for all 5 C A s.t. ies\ . (4) 
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Figure 1: Players' neighbor graphs for 6 players and two different time instances. 

We assume that each player may observe the allocations of a subset of the other 
players at any time, which are termed as the neighbors of the player. The players and 
their neighbors at time t can be represented by a directed graph Qit) = {N,S{t)), with 
the vertex set N and the set £{t) of directed links. A link (i, j) G S{t) exists if player j 
is a neighbor of player i at time t. We always assume that G S{t) for all t, which is 
natural since every player i can always access its own allocation vector. We refer to graph 
Q{t) as a neighbor- graph at time t. In the graph G{t), a player j is a neighbor of player i 
(i.e., {i,j) G £(t)) only if player i can observe the allocation vector of player j at time t. 
Figure [T] illustrates how the players' neighbor-graph may look at two time instances. 

Given the players' neighbor-graph Q(t), each player i negotiates allocations by aver- 
aging his allocation and the allocations he received from his neighbors. More precisely, 
at time t, the bargaining process for each player i involves the player's individual bound- 
ing set Xi[t), its own allocation and the observed allocations x^{t) of some of his 
neighbors j. Formally, we let A/i(t) be the set of neighbors of player i at time t (including 
himself), i.e.. 



where ajj(t) > is a scalar weight that player i assigns to the proposed allocation x^{t) 
of player j and Pxi{t)[-] is the projection onto the player i bounding set Xi{t). The initial 
allocations x*(0), i = 1, . . . ,n, are selected randomly and independently of {v{t)}. 

The bargaining process in ([s]) can be written more compactly by introducing zero 
weights for players j whose allocations are not available to player i at time t. Specifically 
by defining aij(t) = for all j ^ J^i(t) and all t > 0, we have the following equivalent 
representation of the bargaining process: 






for alH G iV and t > 0, 



(5) 



ieA/'(t) 




for alH G and t > 0. 



(6) 



Here, aij{t) = for all j ^ Mi{t) while aij{t) > for j G Mi{t). 
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We now discuss the specific assumptions on the weights cijj(t) and the players' neighbor- 
graph that we will rely on. We let A{t) be the weight matrix with entries aij{t). We will 
use the following assumption for the weight matrices. 

Assumption 3 Each matrix A{t) is doubly stochastic with positive diagonal. Further- 
more, there exists a scalar a > such that 

aij{t) > a whenever aij{t) > 0. 

In view of the construction of matrices A{t), we see that aij(t) > a for j = i and perhaps 
for some players j that are neighbors of player i. The requirement that the positive weights 
are uniformly bounded away from zero is imposed to ensure that the information from 
each player diffuses to his neighbors in the network persistently in time. The requirement 
on the doubly stochasticity of the weights is used to ensure that in a long run each player 
has equal influence on the limiting allocation vector. 

It is natural to expect that the connectivity of the players' neighbor-graphs Q{t) = 
{N, S{t)) impacts the bargaining process. At any time, the instantaneous graph Q{t) need 
not be connected. However, for the proper behavior of the bargaining process, the union 
of the graphs Q{t) over a period of time is assumed to be connected, as given in the 
following assumption. 

Assumption 4 There is an integer Q > 1 such that the graph ^iV, Urig*^ ^ ^i^)^ 
strongly connected for every t > 0. 

Assumptions [3] and |4] together guarantee that the players communicate sufficiently of- 
ten to ensure that the information of each player is persistently diffused over the network 
in time to reach every other player. Under these assumptions, we will study the dynamic 
bargaining process in (|6]). We want to provide conditions under which the process con- 
verges to an allocation in the core of the robust game. Before this, we provide some 
motivating examples in the following section. 

2.2 Motivations 

Dynamic coalitional games capture coordination in a number of network flow applications. 
Network flows model flow of goods, materials, or other resources between different produc- 
tion/distribution sites |2j. We next provide two examples. The first one is more practically 
oriented and describes a supply chain application [6J . The second one is more theoretical 
and re-casts the problem at hand in terms of controlled fiows over hyper-graphs. 

2.2.1 Supply chain 

A single warehouse vq serves a number of retailers Vj, i = 1, . . . ,n, each one facing a 
demand di(t) unknown but bounded by pre assigned values (i™™ G M and dj"^^ G M at any 
time period t > 0. After demand di{t) has been realized, retailer Vj must choose to either 
fulfill the demand or not. The retailers do not hold any private inventory and, therefore, 
if they wish to fulfill their demands, they must reorder goods from the central warehouse. 



7 



Retailers benefit from joint reorders as tliey may sliare tlie total transportation cost K 
(this cost could also be time and/or players dependent). In particular, if retailer Vj "plays" 
individually, the cost of reordering coincides with the full transportation cost K. Actually, 
when necessary a single truck will serve only him and get back to the warehouse. This is 
illustrated by the dashed path (vq, vi, vq) in the network of Figure 2(a) 
reordering is the cost of the unfulfilled demand di{t). 



The cost of not 




(a) Truck leaving vq, serv- 
ing Vl and returning to vq. 




Vl 



V3 



(b) Truck leaving Vq, serv- 
ing vi and V2 , and then re- 
turning to Vq. 




(c) Truck leaving Vg, serv- 
ing vi, V2, and V3, and 
then returning to vq. 



Figure 2: Example of one warehouse vq and three retailers vi, V2 and V3. 



If two or more retailers "play" in a coalition, they agree on a joint decision ("everyone 
reorders" or "no one reorders"). The cost of reordering for the coalition also equals 
the total transportation cost that must be shared among the retailers. In this case, 
when necessary a single truck will serve all retailers in the coalition and get back to the 
warehouse. This is illustrated, with reference to coalition {vi,V2} by the dashed path 



Vo, Vl, V2, Vq) in Figure 2(b) , A similar comment applies to the coalition {vi, V2, V3} and 



the path (vq, Vi, V2, V3, vq) in Figure 2(c) The cost of not reordering is the sum of the 
unfulfilled demands of all retailers. How the players will share the cost is a part of the 
solution generated by the bargaining process. 

The cost scheme can be captured by a game with the set = {vi, . . . , v„} of players 
where the cost of a nonempty coalition S" C A^ is given by 



cs{t) = mini K,J2diit) > 
[ ies J 



Note that the bounds on the demand di{t) reflect into the bounds on the cost as follows: 
for all nonempty S N and t > 0, 



mm 



j K, J2 dr I < cs{t) < min | K, dT"" \ ■ (7) 
K i&s ) I ieS J 



To complete the derivation of the coalitions' values we need to compute the cost savings 
vs{t) of a coalition S as the difference between the sum of the costs of the coalitions of 
the individual players in S and the cost of the coalition itself, namely. 



^sit) =^C{i}{t)-Cs{t). 
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V3 


V4 


V5 


V6 


V7 


s 


{1} 


{2} 


{3} 


{1,2} 


{1,3} 


{2,3} 


{1,2,3} 



Table 1: Vertex-coalition correspondence for a 3-player coalitional game. 



Given the bound for cs{t) in ([T]), the value vs{t) is also bounded, as given: for any 
^ C and t > 0, 



vsit) < ™^ ^^"""^ ~ \ 5Z r 

i&s I ies J 



Thus, the cost savings (value) of each coalition is bounded uniformly by a maximum value. 



2.2.2 Network controlled flows 

Here, we provide a more theoretical example where the bargaining process provides a 
way for the nodes of a hyper-graph to agree on the edge controlled flows [2j. Consider 
the hyper-graph Ti = {V,E} with the vertex set V = {vi,...,Vm} and the edge-set 
E = {ei, . . . , e„}. The vertex set V has one vertex per each coalition, while the edge-set 
E has one edge per each player. An edge e^ is incident to a vertex Vj if the player i is 
in the coalition associated with v^. For a 3-player coalitional game, the vertex-coalition 
correspondence is shown in Table [T] The corresponding hyper-graph is shown in Figure [3j 



■^1 

• / 


^ • 






• ( 










• 



Figure 3: Hyper-graph "H = {V^, -E} for a 3-player coalitional game. 



The incidence relations are described by a matrix i?^ = [e'^j^cAr G M"*^" whose rows 
are the characteristic vectors of all coalitions S N. The flow control reformulation 
arises naturally by viewing an allocation value ai{t) of player i as the flow on the edge e^ 
associated with player i. The demand dj{t) at a vertex Vj is the value vs{t) of the coalition 
S associated with the vertex v^ . In view of this, an allocation in the core translates into 
satisfying in excess the demand at the vertices, or formally: 

a{t) e C{v{t)) ^ Bnait) > d{t). 

In this case, the efficiency condition X]jeAr^«('^) — '"Nit) corresponds to the equality 
constraint J2ieN ^iit) = dm{t). When dmif) is a constant, say dmif) = / for an / > 0, the 
resulting constraint 'Yl,i<^N ^iif) = f can be interpreted as a total flow constraint, stating 
that the total flow in the network has to be equal to the given value /. 
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In this framework, the bargaining process is an algorithmic and distributed mecha- 
nism that ensures the players reach an agreement on controlled flows satisfying uncertain 
demand at the nodes. 

2.3 Preliminary Results 

In this section we derive some preliminary results pertinent to the core of the robust game 
and some error bounds for polyhedral sets applicable to the players' bounding sets Xi(t). 
We later use these results to establish the convergence of the bargaining process in (|6]). 

In our analysis we often use the following relation that is valid for the projection 
operation on a closed convex set X C M": for any w G M" and any x G X, 

||PxM - xf < \\w - xf - ||PxH - , (8) 

This property of the projection operation is known as a strictly non-expansive projection 
property (see [7], volume II, 12.1.13 Lemma on page 1120). 

We next prove a result that relates the distance dist{x, C{ri)) between a point x and 
the core C{ri) with the distances dist(x, Xj(r7)) between x and the bounding sets Xj(?7). 
This result will be crucial in our later development. The result relies on the polyhedrality 
of the bounding sets Xi{r]) and the core C{r]), as given in ([T]) and (|2| respectively, and 
a special relation for polyhedral sets. This special relation states that for a nonempty 
polyhedral set P = {x G M" | a'^x < bg, i = 1, . . . ,r}, there exists a scalar c > such that 

r 

dist(x, V) < dist(x. He) for all x G M", (9) 

e=i 

where He = {x E M" | a'^x < be} and the scalar c depends on the vectors ae,i = 1, . . . , r 
only. Relation ^ has been established by Hoffman [12] and is known as Hoffman hound. 

Aside from the Hoffman bound, in establishing the forthcoming Lemma [T| we also use 
the fact that the square distance from a point x to a closed convex set X contained in an 
affine set H is given by 

dist2(x,X) = ||x-Pff[x]||2 + dist^(Pff[x],X), (10) 

which is illustrated in Figure |4j 

Now, we are ready to present the result relating the values dist^(x, C{rj)) and dist^(x, Xi{r])). 

Lemma 1 Let < N,-)] > be a TU game with a nonempty core C{ri). Then, there is a 
constant /x > such that 

n 

dist2(x, C(r/)) < /i ^ dist^(x, Xi{r])) for all x G M", 

1=1 

where /i depends on the collection of vectors {es \ S C N, S ^ 0}, where each es is the 
projection of es on the hyperplane if = {x G M" | e'^x = rj^}. 
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Figure 4: Projection on a set X contained in an affine set H. 



Proof. Since the hyperplane H contains the core C{ri) (see ([2])), by relation (10) we have 

dist^ {x, C {r])) = \\x- PH[x]f + dist^{PH[x],C (ri)) for all s G M". (11) 

The point and the core C{7]) lie in the n — 1-dimensional affine set H. By applying 

the Hoffman bound relative to the affine set H (cf. Eq. (19))), we obtain 



dist(Pj,[x],C(r7)) <c 5^dist(PHM,if ni/s), 



ScN 



where the summation is over nonempty subsets S, Hs = {x G | e'gX > rjs}, while 
the constant c depends on the collection {e^, S C A^} of projections of vectors on the 
hyperplane H for S G N. Thus, it follows 

dist\PH[x],C{v)) < ('5^dist(PHN,/fni/5)) 

\SCN J 

< c2(m-l) ^dist2(PH[x],ffni/5), (12) 

ScN 

where m is the number of nonempty subsets of and the last inequality follows by 
(Sj=i ^jY — ^ Sj=i which is valid for any finite collection of scalars a^, j = 1, . . . , £ 
with i>l. Combining Eq. (12) with equality (11), we obtain for any x G M", 

dist\x,C{7])) < \\x-PH[x]f + c\m-l)^dist\PH[x],Hr]Hs) 

ScN 

< ci 5^ (||a;-Pj^[a;]f + dist'(PHN,/7nif5)), 



ScN 

r2l 



where Ci = maxjl, c {m — 1)}. Since the set H is affine, in view of relation (10) we have 
\\x - Pff[a;]||2 + dist^(Pj^[a;], if fl Hg) = dist^(x, H fl Hs), implying that for any x G M", 

dist^(x, C{ri)) < ci ^ dist^(x, HnHs). 

ScN 

From the preceding relation, it follows for any x G M", 

dist^(x, C{r])) <ci^\S\ dist^(x, HnHs), (13) 

5c 
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where \S\ denotes the cardinahty of the coahtion 5*. Note that 

\S\ dist2(x, Hr\Hs)=J2J2 dist^(x, H n Hg) 
ScN scN ies 

= Ef E dist'(x,i/ni/5)) . (14) 

We also note that Xiirf) G H (1 Hs for each nonempty S G N and i E S, which follows 
by the definition of Hs and relation ([T]). Furthermore, since dist(x,y) < dist(x,X) for 
any a; G and for any two closed convex sets X,Y G M" such that X G Y, it follows 
that for all a; G M", 

dist(a;, H n Hg) < dist(x, Xi{ri)). (15) 



By combining relations (13)-(15), we obtain 



dist'(x,C(77)) < c^J2\ Yl dist2(a;,^.(^)) 

«=1 \{5cAf|jGS'} 



ciK dist^(x, Xi{r])), 

i=l 



where k is the number of coalitions S that contain player i, which is the same number for 
every player [k does not depend on i). The desired relation follows by letting fi = cik, 
and by recalling that ci = max{l, c^(m — 1)} and that c depends on the projections es of 
vectors 65, S' C A^, on the hyperplane H. m 

Note that the scalar n in Lemma [T] does not depend on the coalitions' values rjs for 
S N. It depends only on the vectors e^, S <^ N, and the grand coalition value rj^. 
As a direct consequence of Lemma [ij we have the following result for the instantaneous 



game < N,v{t) > under the assumptions of Section 2.1 



Lemma 2 Let Assu'mptions\I\ and\^ hold. We then have for all t > 0, 

n 

dist^(x, C(f (t))) <f^Yl dist^(x, Xi{t)) for all x G M", 



i=l 



where C{v{t)) is the core of the game < N,v{t) >, Xi(t) is the bounding set of player i, 
and fi is the constant from Lemma [I| 

Proof. By Assumption [2| we have that the core C{v^^^) is nonempty. Furthermore, 
under Assumption [l| we have C{v^^^) C C{v{t)) for all t > 0, implying that the core 
C{v{t)) is nonempty for all t > 0. 

Under Assumption [l| each core C{v{t)) is defined by the same affine equality corre- 
sponding to the grand coalition value, e'j^x = f^*^^. Moreover, each core C{v{t)) is defined 
through the set of hyperplanes Hs(t) = {x G M" | e'gX > vs(t)}, S G N, which have time 
invariant normal vectors 65, SON. Thus, the result follows from Lemma [T| ■ 
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3 Convergence to Core of Robust Game 



In this section, we prove convergence of the bargaining process in (|6| to a random allo- 
cation that lies in the core of the robust game with probability 1. We find it convenient 
to re-write the bargaining protocol ^ by isolating a linear and a non-linear term. The 
linear term is the vector w^{t) defined as: 



for alH e and t > 0. 



(16) 



Note that w^{t) is linear in players' allocations x^{t). The non-linear term is the error 

e\t) = Px^^,)[w\t)]-w\t). (17) 



Now, using (16) and (17), we can rewrite protocol (|6]) as follows: 

x\t + 1) = w\t) + e\t) for alH G and t > 0. (18) 

Recall that the weights aij{t) are nonnegative and such that ajj(t) = for all j ^ Niit). 
Also, recall that A{t) is the matrix with entries aij{t), which governs the construction of 
the vectors w*(t) in ( [l6| ). 

The main result of this section shows that, with probability 1, the bargaining proto- 
col (16)-(18) converges to the core C(f™'^^) of the robust game < N,v^^^ >, provided 
that v{t) = v'^^^ happens infinitely often in time with probability 1. To establish this we 
use some auxiliary results, which we develop in the next two lemmas. 

The following lemma provides a result on the sequences x'^{t) and shows that the errors 
e*(t) are diminishing. 

Lemma 3 Let Assumptions^ and^ hold. Also, assume that each matrix A{t) is doubly 
stochastic. Then, for the bargaining protocol (16) -(18), we have 

(a) The sequence {^2^=1 \\x'^{t + 1) — xp} converges for every x G C{v'^'^^). 

(b) The errors e*(t) in (17) are such that Xlt^o X]j=i ll^^l'^) P < particular, 
limt^oo ||e*(t)|| = for all i G A^. 



Proof. By x'^{t+l) = Pxi(t) [^*(^)] and by strictly non-expansive property of the Euclidean 
projection on a closed convex set Xi{t) (see ([s])), we have for any z G A^, t > and 
X G Xi{t), 



\x\t + l)-xf < \\w\t) 



x\ 



(19) 



Under Assumptions [T] and [2| the core C(t'™^^) is contained in the core C{v{t)) for all t > 0, 
implying that C(t;"^^^) C C{v{t)) for all t > 0. Furthermore, since C{v{t)) = n^=iX,(t), 
it follows that C(t;"^^^) C Xi{t) for alH G A^ and t > 0. Therefore, relation holds 
for all X G C(w™^^). Thus, by summing the relations in (19) over i & N, we obtain for all 
i > and a; G C(t.'"^^), 



n 

E 

i=l 



\x'{t + l)-xf < 5]]||w^(t) 



x\ 



(20) 



i=l 



13 



By the definition of w^{t) in (16), using tlie stocliasticity of A{t) and the convexity of the 
squared norm, we obtain 



— x\ 



i=l 



i=l 



X 



j=i \i=i 



\x^it) — x\ 



By the doubly stochasticity of A(t), we have J27=i^iji^) ~ ^ every j, implying 
^r=i ll""^*!^) — Sr=i substituting this relation in (20), we arrive at 



J2\\x\t+i) 



-x\r < 



x\ 



i=l 



1=1 



E 

i=l 



(21) 



The preceding relation shows that the scalar sequence {J^^^i + 1) — xW^} is non- 
increasing for any given x G C{v™^^). Therefore, the sequence must be convergent since it 
is nonnegative. Moreover, by summing the relations in (21 ) over t = 0, . . . , s and taking 
the limit as s — )■ cxd, we obtain 



t=0 i=l 



1=1 



which implies that limt_!.oo e*(t) = for all i & N. m 

In our next result, we will use the instantaneous average of players allocations, defined 
as follows: 



1 

y{t) = -^^x^it) foralH>0. 



The result shows that the difference between the bargaining payoff vector for any 
player i and the average y(t) of these payoffs converges to as time goes to infinity. The 
proof essentially uses the line of analysis that has been employed in |T7j, where the sets 
Xi{t) are static in time, i.e., = Xj for all t. In addition, we also use the rate result 

for doubly stochastic matrices that has been established in [T5] . 



Lemma 4 Let Assumptions^ and^ hold. Suppose that for the bargaining protocol (16)- 
(18) we have 

= for all i G N. 



hm \\w\t)-y{t)\\=Q. 

t— >oo 



lim ||e' 

Then, for every player i E N we have 

lim \\x\t)-y{t)\\ = 

Proof. For any t > s > 0, define matrices 

^{t, s) = A{t)A{t - 1) • ■ ■ A(s + l)A{s) 



with t) = A(t). Using the matrices and the expression for in (18), we 

can relate the vectors x^{t) with the vectors a;*(s) at a time s for < s < t — 1, as follows: 



X 



{t) = J2mt-l,s)].,x={s)+ E [J2l^{t-hr)],,e^{r-l)]+e\t-l). (22) 

j=l r=s+l \j=l 
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Under the doubly stochasticity of the matrices A(t), using y(t) = ^ jyj=iX^{t) and rela- 



tion (22), we obtain 



t / n 



= ^^^^ " ^) ) alH > s > 0. 



(23) 



i=i 



r—s+l \j=l 



By our assumption, we have limt_j.oo = for all i. Thus, for any e > 0, there is an 

integer s > such that ||e*(t)|| < e for all t > s and all i. Using relations (22) and (23) 
with s = s, we obtain for all i and t > s + 1, 

\ ft/ 

i=i 

t—ln 1 " 

+ E E - 1'^)]^^- - -)^'(^ - 1) + d^'^ - 1) - - - 1)) 

r=s+l j=l 



i-l n 



Ix-'(s) 



r=s+l j=l 



\eHr 



1 " 

\t-m + -Y.\\^Kt-m- 



Since ||e*(t)|| < e for alH > s and all i, it follows that 



i-l n 



\\At)-ym<Y.\m-^.s)],,-]\ \w{m+^ E E|['^(^-i'*-^ 



r=s+l j=l 



2e. 



Under Assumptions [3] and |4| the following result holds for the matrices s), as 
shown in [ll] (see there Corollary 1): 



n 



< 1 



a 



for alH > s > 0. 



Substituting the preceding estimate in the estimate for ||a;*(t) — y{t)\\, we obtain 



\x\t)-y{t)\\ < 1 



a 

4^2 



Letting t — )■ oo, we see that 



|x''(sl 



+ ne 5^ 1 



oo 

limsup ||x*(t) - ?/(t)|| < ne (l 



r=s+l 



r=s+l 



a 

4^2 



4n2 



+ 2e. 



+ 2e. 



r t-r ~| _2 

Note that (l — < 00, which by the arbitrary choice of e yields 



lim \x\t) - y{t)\ = for all i G N. 

t—^OD 
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Now, we focus on ^"^^ [["^^l^) ~ Since w\t) = Yl^=i O'ijif)^^'^) since A{t) 

is stochastic, it follows 

n n n 

1=1 i=l j=l 

Exchanging the order of the summations over, and then using the doubly stochasticity of 
A{t), we have 

n n / n \ n 

\\wKt) - ym < [Y.^^At) \\x'it)-ym = y \\x^{t)-ym. 

1=1 j=l \i=l J j=l 

Since limt_i.oo lla^-'Xt) — yit)\\ = for all j, it follows that XlILi ~ ~^ 0) ^^^^ 

implying \\w'^{t) — y{t)\\ ^ for all i G iV. ■ 

Note that Lemma |4] captures the effects of the matrices A{t) that represent players' 
neighbor-graphs. At the same time. Lemma |3] is basically a consequence of the projection 
property only. So far, the polyhedrality of the sets Xi{t) has not been used at all. We now 
put all pieces together, namely Lemma [2] that exploits the polyhedrality of the bounding 
sets Xi(t), Lemma [s] and Lemma |4| This brings us to the following convergence result for 
the robust game < N, v^^^ > . 

Theorem 1 Let Assumptions [I]-^ hold. Also, assume that 

Prob {v{t) = i.o.} = 1, 

where i.o. stands for infinitely often. Then, the players allocations x*(t) generated by bar- 
gaining protocol (16) -(18) converge with probability 1 to an allocation in the core C(f™^^), 
i.e., there is a random vector x G C(f™'*^) such that with probability 1, 

lim \\x\t) - x|| = for all i e N. 

t—^CX) 

Proof. By Lemmajsj for each player i & N, the sequence {Yl^=i is convergent 

for every x G C{v™^^) and the errors e*(t) are diminishing, i.e., ||e*(t)|| — )■ 0. Then, by 
Lemma [4] we have ||x*(t) — y{t)\\ — )■ for every i. Hence, 

— 2; 11} is convergent for every x G C('L'™^^). (24) 

We want to show that {y{t)} is convergent and that its limit is in the core C(f™'^^) with 
probability 1. For this, we note that since x\t + 1) G Xi(t), it holds 

n n 

Y dist^ {y{t + 1), Xi(t)) < Y \W + l)-x\t + l)f for all t > 0. 

1=1 i=l 

The preceding relation and ||x*(t) — y{t)\\ — )■ for alH G (cf. Lemma E| imply 



lim V dist^ (y(t + l),Xi(t)) = 0. 



t—>-oo 

1=1 
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Under Assumptions [T] and [2} by Lemma [2] we obtain 

n 

dist^ {y{t + 1), C{v{t))) <^lY^ dist^ {y{t + l),Xi{t)) for all t > 0. 



1=1 



By combining the preceding two relations we see that 

lim dist^ {y{t + 1), C{v{t))) = 0. (25) 



t—^oo 



By our assumption, the event {v{t) = ■y™'^^ infinitely often} happens with probabil- 
ity 1. We now fix a realization {v^(t)} of the sequence {v(t)} such that v^(t) = v"^^^ holds 
infinitely often (for infinitely many t's). Let {tk} be a sequence such that 

v^{tk) = v'^'''' for all A;. 

All the variables corresponding to the realization {v^^it)} are denoted by a subscript u. By 
relation ( [24) ) the sequence {Vuiii)} is bounded, therefore {yuiitk)} is bounded. Without loss 
of generality (by passing to a subsequence of {t^} if necessary), we assume that {Vuiitk)} 
converges to some vector y^^, i.e., 

lim ?/^(tfc) = yu,. 

k—^oo 



Thus, the preceding two relations and Eq. (25) imply that y^ G C{v™'^^). Then, by 
relation (24), we have that {||?/a;(i^) — y^ll} is convergent, from which we conclude that y^ 
must be the unique accumulation point of the sequence {y^(t)}, i.e., 

hmy^{t)=y^, y^eC{v^^^). 

t— ^oo 

This and the assumption Prob {v{t) = ■y™'^^ i.o.} = 1, imply that the sequence {y{t)} 
converges with probability 1 to a random point y G C(v™'"). Since by Lemma |4] we have 
||x*(t) — yit) II — 7- for every i, it follows that the sequences {x\t)}, i = 1, . . . ,n, converge 
with probability 1 to a common random point in the core C(f™'*^). ■ 



4 Dynamic Average Game 

When the core of the robust game is empty, the core of the instantaneous average game can 
provide a meaningful solution under some conditions on the distribution of the functions 
v{t). In what follows, we focus on the instantaneous average game associated with the dy- 
namic TU game < A^, {v{t)} >. In the next sections, we define the instantaneous average 
game, we introduce a bargaining protocol for the game and investigate the convergence 
properties of the bargaining protocol. 

4.1 Average Game and Bargaining Protocol 

Consider a dynamic TU game < A^, {v{t)} > with each v{t) being a random characteristic 
function. With the dynamic game we associate a dynamic average game < N, {v{t)} >, 
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where v(t) is the average of the characteristic functions f (0), . . . ,v{t), i.e., 



1 



^v{k) foralH>0. 



t + 1 



An instantaneous average game at time t > is the game < N,v{t) >. We let C{v{t)) 
denote the core of the instantaneous average game at time t and let Xi{t) denote the 
bounding set of player i for the instantaneous game < N,v{t) >, i.e., for all i E N and 
t > 0, 

Xi{t) = {xeW\ e'^x = VN{t), e'sx > vsit) for a\\ S C N such that i E S} . (26) 

Note that Xj(0) = Xi{0) for alH G since 'i;(0) = "^(0). In what follows, we assume that 
Xi{t) are nonempty for alH G A^ and all t > 0. 

In this setting, the bargaining process for the players is given by 



where aij(t) > is a scalar weight that player i assigns to the proposed allocation x^(t) 
received from player j at time t. The initial allocations x*(0), i E N, are selected randomly 
and independently of {v(t)}. Regarding the weights aij{t), recall that these weights are 
reflective of the players' neighbor-graph: aij{t) = for all j ^ J\fi{t), where J\fi{t) is the 
set of neighbors of player i (including himself) at time t, while we may have aij{t) > 
only for j G Afi{t). 

4.2 Assumptions and Preliminaries 

In this section, we provide our assumptions for the average game, and discuss some aux- 
iliary results that we need later on in the convergence analysis of the bargaining protocol. 
Regarding the random characteristic functions v{t) we use the following assumption. 

Assumption 5 The sequence {v{t)} is ergodic, i.e., with probability 1, we have 



Furthermore, f Af(t) = v^^^^ for all t with probability 1. 

Assumption [5] basically says that the grand coalition value fAr(t) is constant with proba- 
bility 1. Note that Assumption |5] is satisfied, for example, when {vs{t)} is an independent 
identically distributed sequence with a finite expectation E[f5(t)] for all S G N. 

We refer to the TU game < A^, y™'^^'^ > as average game, which is well defined under 
Assumption [s} We let C (-y™'^'^") be the core of the average game and Xi be the bounding 
set for player i in the game, i.e.. 




for alH G A^ and t > 0, 



(27) 



lim v{t) = V 



,mean 



with v'^''^'' G M' 



mean 



) = {x G M" I e'j^x = V 



mean 




•s 



mean 



for all nonempty S C A^} , 



18 



Xi = {xeW\ e'^x = v'^^'''', e'sx > v^'"''' for al\ S C N such that i e S} . (28) 

Note that the average core C(f™'^'^") hes in the hyperplane if = {x G | e'j^x = v'^'^^^}. 
Hence, the dimension of the core is at most n — 1. We will in fact assume that the 
dimension of the average core C(f™'^'^'^) is — 1, by requiring the existence of a point z in 
the core such that all other inequalities defining the core are satisfied as strict inequalities. 
Specifically, we make use of the following assumption. 

Assumption 6 There exists a vector z G such that 

e'gZ > v^'"^'^ for all nonempty S C N. 



Assumption |6] basically says that z is in the relative interior of the core C(f™''^°) of the 
average game and that the core C(f™'^'^'^) has dimension n — 1. This assumption will be 



important in establishing the convergence of the bargaining protocol (27). In particular, 
the following result will be important, which is an immediate consequence of Assumption |6] 
and the polyhedrality of the cores C{v{t)). 

Lemma 5 Let Assumptions\5\ and\^ hold. Then, with probability 1, for every z in relative 
interior o/C(t'™'^^") there exists tz large enough such that z is in the relative interior of 
C{v{t)) for all t > tz with probability 1. 



Proof. Let z be in the relative interior of C{v™^^'^) which exists by Assumption |6} Thus, 
e'j^z = vj^^^"^ and e'gZ > ti™''^" for all S C N. By Assumption [5| with probability 1 we 
have VN{t) = v'^'''''' and vsit) w^^'^'^ for S C N. Hence, %(t)'2 = {v'^"''''')' z for all t 
with probability 1. Furthermore, there exists a random time tz large enough so that with 
probability 1, 

e'gZ > vs(t) for all S C and all t >tz, 

implying that z is in the relative interior of C{v(t)) for all t > tz with probability 1. ■ 
Lemma [s] shows that the sets C{v{t)) and C{v^'^^^) have the same dimension for large 
enough t with probability 1. In particular, this lemma implies that the cores C{v{t)) are 
nonempty with probability 1 for all t sufficiently large. 

Aside from Lemma |5| in our convergence analysis of the bargaining protocol, we make 
use of two additional well-known results. One of them is the super-martingale convergence 
theorem due to Robbins and Siegmund [i21j (it can also be found in [18], Chapter 2.2, 
Lemma 11). 

Theorem 2 Let {Vk}, {gk}, o,nd {hk} be non-negative random scalar sequences. Let Fk 
be the a -algebra generated by Vi, . . . ,Vk, gi, . . ■ gk, hi, . . . hk. Suppose that almost surely, 

E[Vfc+i I Fk] <Vk- gk + hk for all k, 

and Xlfc ^fc < oo almost surely. Then, almost surely both the sequence {Vk} converges to 
a non-negative random variable and < 

The other well-known result that we use is pertinent to two nonempty polyhedral sets 
whose description differs only in the right-hand side vector. The result can be found in [7] , 
3.2.5 Corollary, pages 258-259. 
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Lemma 6 Let Pb and P^ be two polyhedral sets given by 

Pb = {x e M" I Bx < b}, P-^ = {x eW' \ Bx < b}, 

where B is an m x n matrix and b,b & W^. Then, there is a scalar L > such that for 
every b,b & for which 7^ and Pi 7^ 0, we have 

dist(x, Pi) < — 6|| for any x G Pb, 

where the constant L depends on the matrix B. 

For two polyhedral sets Pb and P^ as in Lemma |6] we also have 

dist(x, Pb) < dist(x, Pi) + L\\b - b\\ for any x eR"^ (29) 

(see Rockafellar [22], 4.34 Lemma on page 132). 

4.3 Convergence to Core of Average Game 

Here, we show the convergence of the bargaining protocol to the core of the average 



game. In our analysis, we find it convenient to re- write the bargaining protocol (27) in 



an equivalent form by separating a linear and a non-linear term. The linear term is the 
vector w''(t) given by 



(t) = ^ aij{t)x^{t) for alH G and t. 



(30) 



The non-linear term is expressed by the error 

e\t) = Pji^^,)[w\t)]-w\t). 



Now, using relations (30) and (31), we can rewrite (27) as follows: 



x\t + l) = w\t) + e'{t) 



for alH G and all t > 0. 



(31) 



(32) 



We first show some basic properties of the players' allocations by using the preceding 



equivalent description of the bargaining protocol (27). These properties hold under the 



doubly stochasticity of the weights aij{t) that comprise the matrix A{t). 

Lemma 7 Let Assumptions\^and \^ ho ld. Also, let the matrices A{t) be doubly stochastic. 
Then, for bargaining protocol (30)-(|32l), we have with probability 1: 



(a) The sequence {Yl^=i + 1) ^ -^^IP} converges for every z in the relative interior 
o/C(w™). 



(b) The errors e*(t) in (17) are such that Xlt^o X]j=i 11^' 
lim^^oo ||e*(t)|| = for all i G A^. 



< 00. In particular. 
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Proof. Let rinty denote the relative interior of a set Y. Let z G rintC [v^'^'^'^) be arbitrary 
and fixed for the rest of the proof. By Lemma [5| there exists large enough such that 
z e rintC(t;(t)) for all t > with probability 1. Since C{v{t)) = n"^iXj(t), it follows 
that z e Xi{t) for alH G and t>tz with probability 1. 

Fromx*(t+l) = w^{t)+e^{t) (cf. (32)), the definition of e*(t) in (31 ), and the projection 



property given in Eq. ([8j), we have with probability 1, for i E N and all t >tz, 

\\x\t + 1) - zf < \\w\t) - - ||r^^-Mi2 



Summing these relations over i E N, and using w^t) = YTj=i'^ij^\^)^ the convexity of 
the norm and the stochasticity of A{t), we obtain 



+ 1) - ^ir < a^MWit) - -II - 

4 = 1 1=1 j = l j=l 

Exchanging the order of summation in ^^^i ^"=i C'ij{'t)\\^'' (t) ~ -^IP and using the doubly 
stochasticity of A{t), we have for all t > tz with probability 1, 

n n n 

1=1 j=i j=i 

Applying the super-martingale converge theorem (Theorem [2]) (with an index shift), we 
see that the sequence {J2^=i \\x''(t + 1) — zp} is convergent and Ylt^t^ ^i=i ^ 
with probability 1. Hence, the result in part (b) follows. ■ 

We observe that Lemma |4] applies to protocol (30)-(32) in view of the analogy of the 
description of the protocol in ([l6|)-(18) and the protocol in (30)-(32). We will re-state 
this lemma for an easier reference, but without the proof since it is almost the same as 
that of Lemma |4] (the proof in essence depends mainly on the matrices A(k)). 



Lemma 8 Let Assumptions^ and^ hold. Suppose that for the bargaining protocol (30)- 
(32) we have with probability 1, 

lim ||e*(t)|| = for alii EN. 

t—^oo 

Then, for every player i E N we have with probability 1, 

lim \\x\t) - y{t) II = 0, lim \\w\t) - y{t) \\ = 0, 

where yit) = ^ T.]=iX^it). 

We are now ready to show the convergence of the bargaining protocol. We show this 
in the following theorem by combining the basic properties of the protocol established in 
Lemma [7] and using Lemma [8j 



Theorems Let Assumptions \3^^ hold. Then, the bargaining protocol (30) -(32) con- 
verges to a random allocation that lies in the core C{v"^°''^'^) of the average game with 
probability 1, i.e., there is a random vector z E C{v'^'^^^) such that with probability 1, 

lim \\x\t) - 5|| = for all i E N. 

t— >oo 
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Proof. By Lemma [7| with probability 1, the sequence {Y^^=i \\^^{t) ~ ^IP} is convergent 
for every z in the relative interior of and ||e*(t)|| — > for each player i ^ N. By 

Lemma [8} with probability 1 we have 

lim ||a;*(t) - = for every i e A^. (33) 

t—^oo 

Hence, with probability 1, 

{||?/(t) — z\\} is convergent for every z G rintC(t'™'"^°). (34) 

We next show that {y{t)} has accumulation points in the core C(f™^^") with proba- 
bility 1. Since x'{t + l) e Xi{t), it follows 

n n 

^ dist^ {y{t + l),Mt)) < E + ^)-^\t+ (35) 

i=l i=l 



Relations (35) and (33) imply 



lim V dist^ (y{t + l),Xi{t)) = with probability L (36) 



i=l 



Let i ^ N he arbitrary but fixed and let x G Xj be arbitrary. Note that we can write 

bit + 1) - x\\ < Wvit + 1) - Pjt^i^tMt + 1)]|| + WPx.itMt + 1)] - x\\. 

By taking the minimum over x E Xi and using \\y{t + 1) — Px.(j)[?/(t + 1)]|| = dist{y{t + 
l),Xi{t)), we obtain for alH G and t > 0, 

dist(y(t + l),Xi) < dist{y{t + l),Xi{t)) + dist (Px,(t)b(t + 1)],^*) • 
The bounding sets Xi are nonempty by Assumption p] and the fact C(t;™°^°) C Xj for all i, 



while Xi{t) are assumed nonempty (see the discussion after relation (26)). Furthermore, 
since Xi and Xi(t) are polyhedral sets, by Lemma |6] it follows that for a scalar Lj > 
and all t >0, 

dist (Px,wb(t + 1)],X,) < L4v{t) - v~\\. 
Therefore, for alH G X and alH > 0, 

dist(y(t + 1),X,) < dist(y(t + l),X,(t)) + L,\\v{t) - v~\\. 

By letting t — )• oo and using relations (36) and v{t) — v""^^^ (Assumption [s]) , we see that 
for alH G X with probability 1, 

lim dist(?/(t + l),Xi) = 0. (37) 



In view of relation (34), the sequence {y{t)} is bounded with probability 1, so it 
has accumulation points with probability 1. By relation (37), all accumulation points of 
{y{t + l)} lie in the set Xj for every i G X with probability 1. Therefore, the accumulation 
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points of {y(t)} must lie in the intersection Di^N^i with probability 1. Since Hi^N^i = 
C(f™''^°), we conclude that all accumulation points of {y{t)} lie in the core C{v"^'^'^'^) with 
probability 1. Furthermore, according to relation (34) we have that, for any point z G 
rintC {v'^'^^^) , the accumulation points of the sequences {y{t)} are at the same (random) 
distance from z with probability 1. Since the accumulation points are in the set C(f™^'^°), 
it follows that {y{k)} is convergent with probability 1 and its limit point is in the core 
(j^^mean^ with probability 1. Now, since — y{t)\\ — with probability 1 for all i 

(see ([33|)), the sequences {x^{t)}, i G N, have the same limit point as the sequence {y{t)}. 
Thus, the sequences {x^{t)}, i & N, converge to a common (random) point in C(f™'^^'^) 
with probability 1. ■ 

Theorem [3] shows the convergence of the allocations generated by the bargaining pro- 
tocol in ([30|)-([32|. The convergence relies on the properties of the matrices and the 
connectivity of the players' neighbor graphs, as reflected in Assumptions |3] and |4j It also 
critically depends on the fact that the core of the average game C{v"^^^^) has dimension 
n — 1 and that all bounding sets Xi{t) and, hence the cores C{v{t)), lie in the same hyper- 
plane with probability 1, the hyperplane defined through the constant value of the grand 
coalition. 



5 Numerical Illustrations 

In this section, we report some numerical simulations. We consider coalitional TU games 
with 3 players, so the number of possible nonempty coalitions is m = 7. We consider 
two different scenarios as shown respectively in rows 1 and 11 in Table |2} The columns of 
Table |2] enumerate the coalitions. In each scenario, the characteristic functions vs{t) are 
generated independently with identical uniform distribution over an interval. Specifically, 
we suppose that the values of single player coalitions {1} and {2} are uncertain within 
the given interval. All the other coalitions' values are fixed and equal to zero except for 
the grand coalition, which has value 10 in both scenarios 1 and 11. 

The two scenarios differ in that the core C(f™'^^) of the robust game is nonempty in 
scenario 1 and empty in scenario 11. For scenario 1, we simulate the convergence behavior 



of the bargaining protocol ( 16 )-( 18 ) for the robust game, while for scenario 11, we simulate 
bargaining protocol (30)-(32) for the average game. 

For each scenario, we run 50 different Monte Carlo trajectories each one having 100 
iterations. The number of iterations is chosen long enough to show the convergence of the 
protocols. All plots include the sampled average and sampled variance for the 50 different 
trajectories that were simulated. Each trajectory in each scenario is generated by starting 
with the same initial allocations, which are given by x^(0) = [10 0]', x^(0) = [0 10 0]', 
and x^(0) = [0 10]'. The sampled average is computed for each time t = 1, . . . , 100, by 
fixing the time t and computing the average value of the 50 trajectory sample values for 
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Table 2: Coalitions' values for the two simulations scenarios. 
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that time. The sampled variance is computed as the variance of the samples with respect 
to their sampled average. 

Regarding the players' neighbor-graphs, we assume that the graphs are deterministic 
but time- varying. The graphs for the times t = 0,1,2 are as follows: player 2 and 
3 connected at time t = (see Figure |5(a)[ ), then player 3 and 1 connected at time 
t = 1 (Figure 5(b)), and finally player 1 and 2 connected at time t = 3 (Figure 5(c)). 
These graphs are then repeated consecutively in the same order. In this way, the players' 
neighbor-graph is connected every 3 time units (Assumption |4] is satisfied with Q = 2). 

The matrices that we associate with these three graphs, are respectively given by: 



AiO) 



1 






1 1 

2 2 

h h 



A{1) 
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These matrices are also repeated in the same order for the rest of the time. Thus, at any 
time t, the matrix A{t) is doubly stochastic, with positive diagonal, and every positive 
entry bounded below by |, so Assumption [3] is satisfied with a = ^. All simulations are 
carried out with MATLAB on an Intel(R) Core(TM)2 Duo, CPU P8400 at 2.27 GHz and 
a 3GB of RAM. The run time of each simulation is around 90 seconds. 



5.1 Simulation Scenario I 



In this scenario, the coalitions' values are generated as given in row I of Table [2j In 
particular, at each time t, the value f{i}(t) is chosen randomly in the interval [4, 7] with 
uniform probability independently of the other times. Similarly, the values f{2}(^) are 
generated in the interval [0,3]. The grand coalition value is fixed to 10 at all times, and 
the other coalition values are 0. With this data, we consider the allocations as generated 
by players z = 1, 2, 3 according to the bargaining protocol in ( 16 )-( 18 ) for the robust game 



as reported in Subsection 5.1.1 We then consider the bargaining protocol in (30)-(32) 



for the average game in Subsection 5.1.2 



5.1.1 Robust game 

For this specific example, the characteristic function v^'^^ for the robust game is obtained 
by considering the highest possible coalition values (see the data in row I of Table [2]), 
which results in -y™^^ = [730000 10]'. The resulting core of the robust game is given by 

C(t;'"^^) = {x e : xi >7,X2> 3, X3 > 0, xi + 0:2 > 0, Xi + X3 > 0, 

X2 + 0, Xi + X2 + Xs = 10}. 



Vl Vl Vl 




(a) (b) (c) 

Figure 5: Topology of players' neighbor-graph at three distinct times t = 0, 1 and 2. 
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We note that this core contains a single point, namely [73 0]'. 

To ensure that v{t) = v^^^ infinitely often, as required by Theorem [l] for the conver- 
gence of the protocol, we adopt the following randomization mechanism. At each time 
t = 1, . . . , 100, we flip a coin and if the outcome is "head" (probability 1/2), the coalitions' 
values V{i}{t) and f{2}(i) are extracted from the intervals [4, 7] and [0, 3], respectively, with 
uniform probability independently of the other times. If the outcome of the coin flip is 
"tail", then we assume that the robust game realizes and take v{t) = v^^^. 

We next present the results obtained by the Monte Carlo runs for the bargaining 



protocol in (16)-(18). An illustration of a typical run with the allocations generated in 
periods t = 0, 1, 2, 3 is shown below: 

t;(0) = [6.8 2.7...10]' a;i(0) = [10 0]' ^^(O) = [0 10 0]' a;3(0) = [0 10]' 

t;(l) = [7 3...10]' x^(l) = [10 0]' x^jl) = [0 5 5]' x^jl) = [0 5 5]' 

t;(2) = [4.4 1.1...10]' xi(2) = [5 2.5 2.5]' x2(2) = [0 5 5]' x3(2) = [5 2.5 2.5]' 

t;(3) = [7 3...10]' xi(3) = [7 1.5 1.5]' x2(3) = [2.5 3.75 3.75]' x='(3) = [5 2.5 2.5]'. 

Recall that the initial allocations of the players are a;^(0) = [10 0]', a;^(0) = [0 10 0]', 
and x^{0) = [0 10]'. At time t = 1, bar gaining involves player 2 and 3 who update 
the allocations respectively as x^(l) = [0 5 5]' and x^(l) = [0 5 5]'. These allocations 
are feasible for their bounding sets so the projections on these sets are not performed. 
At time t = 2, the bargaining involves player 1 and 3 who update their allocations, 
respectively, as x^{2) = [5 2.5 2.5]' and x^(2) = [5 2.5 2.5]'. Again, these allocations 
are feasible for their bounding sets and the projections are not performed. Finally, at 
time t = 3, the bargaining involves player 1 and 2 who update their allocations resulting 
in x^{3) = [7 1.5 1.5]' and x^{3) = [2.5 3.75 3.75]'. Notice that x\3) is obtained after 
player 1 projects onto his bounding set. 

In Figures |6] and [7], we report our simulation results for the average of the sample tra- 
jectories obtained by Monte Carlo runs. Figure |6] shows the sampled average and variance 
of the allocations i = 1,2,3 per iteration t. In accordance with the convergence 

result of Theorem [l| the sampled averages of the players' allocations converge to the 
same point, namely x = [73 0]' which is in the core of the robust game 

(^(^max)_ Figure g 

shows that the sample average and sampled variance of the errors e*(t) converge to 0, as 
expected in view of Lemma [3| 



5.1.2 Average game 

In this numerical example, for scenario I data as given in row I of Table |2| we consider the 



average TU game and its corresponding bargaining protocol (30)-(32). We at first provide 
a sample example and report the outcomes for the first three steps of the algorithm, as 
reported below: 

[0 10]' 
[0 5 5]' 
[5 2.5 2.5]' 
[5 2.5 2.5]'. 

Recalling that the initial players' allocations are x^(0) = [10 0]', x^(0) = [0 10 0]', and 
x^{0) = [0 10]', we note that at time t = 1, bargaining involves player 2 and 3 who 
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[4.6 1.8. 


.10]' 


x\2) = 


[5 2.5 2.5]' 


x\2) 


= [0 5 5]' 


x^{2) 


v{3) = 


[4.7 1.8. 


.10]' 


x\3) = 


[4.7 2.6 2.7]' 
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Figure 6: Plots of the sampled average (left) and variance (right) of players' allocations 
x*(t), 2 = 1, 2, 3 generated by bargaining protocol (16)-(18) for the robust game associated 



with the data in row I of Table |2| Sampled averages of the allocations x*(t) converge to 
the same point x = [73 0]' G C(f™'^^), while sampled variances go rapidly to zero. 





Figure 7: Plots of the sampled average (left) and sampled variance (right) of the errors 
e*(t), i = 1,2,3 for the bargaining protocol ((T6j)-(18) and the robust game associated 



with the data in row I of Table |2| Sampled averages and the variances of the errors e*(t) 
converge to zero. 
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Figure 8: Plots of the sampled averages (left) and variances (right) of players' allocations 



x^{t), i = 1,2, 3, obtained by bargaining protocol (30 )-(32 ) for the average game associated 
with the data in row I of Table [2] Sampled averages converge to point x = [5.6 2.2 2.2]' G 
C(f™®^"), which is the average of the limit points of the 50-sample trajectories. 



update their allocations to a;^(l) = [0 5 5]' and a:^(l) = [0 5 5]', respectively. At time 
t = 2, the bargaining involves player 1 and 3 who update their allocations, respectively, 
to a;^(2) = [5 2.5 2.5]' and x^{2) = [5 2.5 2.5]'. Notice that the obtained allocations 
for t = 1,2 are feasible for the bounding sets and, therefore, no projection is performed. 
Finally, at time t = 3, the bargaining involves player 1 and 2 who update their allocations 
to x^(3) = [4.7 2.6 2.7]' and x^(3) = [2.5 3.75 3.75]', respectively. Here x^{3) results from 
projecting onto the bounding set of player 1. 

For the average game associated with the data in row I of Table [2| we have = 
[5.5 1.5 10]' and the core C{v'^'''"^) given by 

C{v~) = {x eR^ : xi> 5.5, X2 > 1.5, X3 > 0, Xi + X2 > 0, Xi + X3 > 0, 

X2+ X3> 0, Xi + X2 + X3 = 10}. 

In Figures |8] and [9} we depict our simulation results generated by bargaining pro- 



tocol (30)-(|32j) for the average game. Figure |8j shows that the sampled average of the 
allocations x'^{t), i = 1,2,3 converge to a common point x = [5.6 2.2 2.2]' which belongs 
to the core C(f™*'^") of the average game, as guaranteed by Theorem |3} The sampled 
variance does not converge to zero as the common limit point of the allocations x^{t) 
can be different for different runs. Figure [9] demonstrates that the sampled average and 
variance of the errors e*(t), i = 1,2, 3, converge to zero, as predicted by Lemma [Tj^b). 



27 



Id) 2 



> 



\a> -2 



60 
t 



20 30 40 50 BO 70 
t 





10 20 30 40 60 BO 70 80 SO 100 
t 



> 

C- 0.05 - 



J? 0.05 



10 20 30 40 60 BO 70 
t 



10 20 30 40 50 BO 70 
t 



Figure 9: Plots of the sampled average (left) and variance (right) of the errors e*(t), 
i = 1,2,3, for bargaining protocol (30)-([32|) and the average game associated with the 



data in row I of Table ^ The sampled average and variance of the errors e*(t) goes to 0. 



5.2 Simulation Scenario II 



Here, we report the simulation results obtained by the bargaining protocol (30)-(32) for 



the average game corresponding to the data in row II of Table [2j In this case the core of 
the robust game is empty, so we do not consider the robust game. The average game has 
characteristic function y'^'^^^ = [6.5 2.5 010]' and its core is 



C{v~) = {xe 



p3 . 



X 



1 > 6.5, X2 > 2.5, 0:3 > 0, a;i + 0:2 > 0, a;i + X3 > 0, 
X2 + xs> 0, xi + X2 + X3 = 10}. 



Figures TO] and [TT] show the results for the average game obtained in our simulations. In 
Figure 10, we report the sampled averages of the players' allocations a;*(t), i = 1,2,3, 



obtained by bargaining protocol (30)-(32). In accordance with Theorem 3 the players 



allocations converge to an allocation that lies in the core of the average game C(f™'^'^°), 
precisely to the point x = [6.6 2.6 0.8]' G C{v'^^'^^). Here, again, the sampled variance 
of the allocations does not converge to zero as their common limit point is different for 
different runs. Figure 11 shows that the sampled average and variance of the errors e*(t). 



1, 2, 3, converge to zero. 



6 Conclusions 

This article deals with dynamics and robustness within the framework of coalitional TU 
games. With respect to a sequence of TU games, each with a random characteristic 
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Figure 10: Plots of the sampled averages (left) and variances (right) of the players' al- 
locations x*(t), z = 1,2,3 obtained by the bargaining protocol (30)-(32) for the average 



game associated with the data in row II of Table [2] Sampled averages converge to point 
X = [6.6 2.6 0.8]' G C(f™'^^'^), which is the average of the limit points of the 50-sample 
trajectories. 



function, the novelty of the work lies in the design of a decentralized allocation process 
defined over a communication graph of players. The allocation process captures the main 
features of bargaining in a realistic scenario. The proposed bargaining scheme is proven 
to converge, with probability 1, in either a robust game setting or an average game 
setting, under mild assumptions on the communication topology among the players and 
the stochastic properties of the random characteristic function. The key properties that 
distinguish this work from the existing work on dynamic games are: (1) the introduction 
of a time-varying communication graph, termed players' neighbor-graph, over which the 
bargaining protocol takes place; and (2) the distributed bargaining protocol for players' 
allocations updates subject to local information exchange with neighboring players. 
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